ABSTRACT
Existing challenges in surgical education (See one, do one, teach one) as well as the Covid-19 pandemic make it necessary to develop new ways for surgical training. This is also crucial for the dissemination of new technological developments. As today's live transmissions of surgeries to remote locations always come with high information loss, e.g. stereoscopic depth perception, and limited communication channels. This work describes the implementation of a scalable remote solution for surgical training, called TeleSTAR (Telepresence for Surgical Assistance and Training using Augmented Reality), using immersive, interactive and augmented reality elements with a bi-lateral audio pipeline to foster direct communication. The system uses a full digital surgical microscope with a modular software-based AR interface, which consists of an interactive annotation mode to mark anatomical landmarks using an integrated touch panel as well as an intraoperative image-based stereo-spectral algorithm unit to measure anatomical details and highlight tissue characteristics.We broadcasted three cochlea implant surgeries in the context of otorhinolaryngology. The intervention scaled to five different remote locations in Germany and the Netherlands with lowlatency. In total, more than 150 persons could be reached and included an evaluation of a participant's questionnaire indicating that annotated AR-based 3D live transmissions add an extra level of surgical transparency and improve the learning outcome. © 2021 by Walter de Gruyter Berlin/Boston.
ABSTRACT
Introduction Existing challenges in surgical education (See one, do one, teach one) as well as the Covid-19-pandemic make it necessary to develop new ways for surgical training. This is also crucial for the dissemination of new technological developments. As todays live transmissions of surgeries to remote locations always come with high information loss, e.g. depth perception, this work describes the implementation of a scalable remote solution for surgical training, called TeleSTAR, using immersive, interactive and augmented reality elements tested during 3 ENT surgeries. Methods The system uses a full digital surgical microscope (ARRISCOPE, Munich Surgical Imaging, Munich, Germany). The microscope has a modular software augmented reality interface, which consists of an interactive annotation mode to mark anatomical landmarks using an integrated touch panel as well as an intraoperative image-based stereo-spectral algorithm unit to measure and highlight anatomical details and tissue characteristics. The stereo measurement unit computes a complete depth map allowing to perform precise distance measurements by setting markers. In addition, the system comes with a bi-lateral audio pipeline to foster direct communication. Results We evaluated the new educational paradigm during the 3D broadcast of three live AR-based cochlea implant surgeries. The system scaled to five different remote locations with low latency and offering a delayed 2D YouTube stream. In total, more than 150 persons could be trained including healthcare professionals, biomedical engineers and medical students. It was demonstrated by a participants survey, that annotated 3D live transmissions improve the learning outcome by adding an extra level of surgical transparency. Conclusion TeleSTAR provides a highly scalable solution for surgical training solving limitations for current educational programs and during pandemic situations by using an adaptive combination of modular software and hardware modules. Furthermore, it can be used as an intraoperative assistance system by support from remote experts and it can be easily adapted to other surgical domains.